{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Noise\n",
    "\n",
    "This example shows how to get non-perfect measurements/apply noise to the images. \n",
    "The sensor library has also been restructured to an approach, where the sensor class has its own `data` property.\n",
    "Before, the sensor data was a return value of the `BeamNGpy.poll_sensors` class.\n",
    "\n",
    "## Setup\n",
    "\n",
    "The `RandomImageNoise` class is a wrapper class for the `skimage.utils.noise.random_noise` function. \n",
    "Since the installation for this package is not required to use BeamNGpy, it has to be installed manually ro use the noise module.\n",
    "See scikit-image.org for install instructions and a documentation of the possible values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from beamngpy import BeamNGpy, Scenario, Vehicle\n",
    "from beamngpy.sensors import Camera\n",
    "from beamngpy.noise import RandomImageNoise\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up Scenario"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "beamng = BeamNGpy('localhost', 64256)\n",
    "beamng.open()\n",
    "scenario = Scenario('west_coast_usa', 'noise demo')\n",
    "vehicle = Vehicle('test_car', model='etk800')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting Up Noise\n",
    "\n",
    "The Noise class is designed in such a way,\n",
    "that different instances of noise can be combined on top of each other.\n",
    "With `RandomImageNoise` it is possibe to apply noise to depth and/or rgb images of the `Camera` class.\n",
    "\n",
    "In this example we will use three types of noise sensors:\n",
    "\n",
    "1. gaussian, which is the default type of noise used, if no other parameters are specified\n",
    "\n",
    "2. salt and pepper, with more salt then pepper, which is also applied to the depth and colour image\n",
    "\n",
    "3. a camera sensor, where gaussian noise is applied to depth image and poisson noise is applied to the colour image.\n",
    "\n",
    "For the selection of parameters it is important to note, that the noise is applied to an image whose values are in \\[0, 1\\]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pos = (-0.3, 1, 1.0)\n",
    "direction = (0, 1, 0)\n",
    "fov = 120\n",
    "resolution = (512, 512)\n",
    "gauss_cam = Camera(pos, direction, fov, resolution)\n",
    "snp_cam = Camera(pos, direction, fov, resolution)\n",
    "multi_noise_cam = Camera(pos, direction, fov, resolution)\n",
    "\n",
    "gauss = RandomImageNoise(gauss_cam)\n",
    "snp = RandomImageNoise(snp_cam, mode=\"s&p\", amount=0.5, salt_vs_pepper=0.8)\n",
    "multi_noise = RandomImageNoise(multi_noise_cam, colour=False, depth=True, mode=\"gaussian\", mean=0, var=0.2)\n",
    "multi_noise = RandomImageNoise(multi_noise, colour=True, depth=False, mode=\"poisson\")\n",
    "\n",
    "vehicle.attach_sensor('gauss', gauss)\n",
    "vehicle.attach_sensor('snp', snp)\n",
    "vehicle.attach_sensor('multi_noise', multi_noise)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running Simulation and Acquiring Data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scenario.add_vehicle(vehicle, pos=(-717.121, 101, 118.675), rot_quat=(0, 0, 0.3826834, 0.9238795))\n",
    "scenario.make(beamng)\n",
    "\n",
    "beamng.load_scenario(scenario)\n",
    "beamng.step(60)\n",
    "vehicle.poll_sensors()\n",
    "beamng.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Noise Visualization\n",
    "\n",
    "### Gauss "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(1, 2, figsize=(30, 30))\n",
    "ax[0].imshow(np.array(gauss.data['colour']))\n",
    "ax[1].imshow(np.array(gauss.data['depth']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Salt and Pepper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(1, 2, figsize=(30, 30))\n",
    "ax[0].imshow(np.array(snp.data['colour']))\n",
    "ax[1].imshow(np.array(snp.data['depth']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Poisson and Gauss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(1, 2, figsize=(30, 30))\n",
    "ax[0].imshow(np.array(multi_noise.data['colour']))\n",
    "ax[1].imshow(np.array(multi_noise.data['depth']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Original\n",
    "The original data can always be accessed through the data property of the camera sensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.imshow(np.array(gauss_cam.data['colour']))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
